Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build without swift host tools #77815

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

DougGregor
Copy link
Member

Introduce various fixes to the build system, compiler, and standard library to enable building without Swift host tools, to make it possible to bootstrap a Swift compiler on a host that does not have Swift yet (without cross-compiling). The compiler built from only C++ doesn't need to have full functionality (and it doesn't---at least macros and embedded won't work), but it needs to have enough functionality to act as host tools for a full build of the toolchain.

…swift-syntax

When building without swift-syntax, we don't have the ability to handle
macros. However, the concurrency runtime makes use of `#isolation` in
a number of places. To continue supporting building without
swift-syntax, parse `#isolation` directly into the otherwise-unspellable
CurrentContextIsolationExpr but *only* when swift-syntax is not available.
@DougGregor
Copy link
Member Author

The preset comes from #77795

@DougGregor
Copy link
Member Author

preset=buildbot_linux,without_host_swift
@swift-ci Please test with preset Linux Platform

1 similar comment
@DougGregor
Copy link
Member Author

preset=buildbot_linux,without_host_swift
@swift-ci Please test with preset Linux Platform

Based on a patch from Evan Wilde, introduce a new preset that disables
the features that require Swift host tools. This involves disabling
early swift-syntax (used in the compiler for macros et al) and early
swift-driver.
@DougGregor
Copy link
Member Author

preset=buildbot_linux,without_host_swift
@swift-ci Please test with preset Linux Platform

These annotations require optimization passes that are written in
Swift, so we need a host Swift compiler to properly process them. Use
`hasFeature(Macros)` as a way to gate the annotations.
@DougGregor
Copy link
Member Author

preset=buildbot_linux,without_host_swift
@swift-ci Please test with preset Linux Platform

@DougGregor
Copy link
Member Author

Next failure is...

Begin Error in Function: '$sSysE15_toUTF16IndicesySnySS5IndexVGSnySiGFSS_Tg5'
Found an operand with a value that is not compatible with the operand's operand ownership kind map.
Value:   %181 = unchecked_bitwise_cast %177 : $String to $Substring // user: %182
Value Ownership Kind: unowned
Instruction:
     %181 = unchecked_bitwise_cast %177 : $String to $Substring // user: %182
->   %182 = begin_borrow %181 : $Substring
Constraint: <Constraint Kind:any LifetimeConstraint:NonLifetimeEnding>
End Error in Function: '$sSysE15_toUTF16IndicesySnySS5IndexVGSnySiGFSS_Tg5'
Found ownership error?!

@DougGregor
Copy link
Member Author

preset=buildbot_linux,without_host_swift
@swift-ci Please test with preset Linux Platform

@DougGregor
Copy link
Member Author

Trying something hideous, unless @eeckstein has any better ideas for the last failure?

@finagolfin
Copy link
Contributor

Is this just an experiment to see how far you get or do you really expect to get bootstrapping to work again? It's been a year since the CI switched over to only using a prebuilt Swift 5.8.1 compiler to build the trunk compiler, so I doubt you can get all those dependencies weeded out again.

Even if you get a basic bootstrap C++-only compiler built again, it may no longer work to build the full compiler with swift-syntax. For example, I just successfully cross-compiled the trunk compiler for Android for the first time in nine months, and I initially tried working around a recent swift-driver issue on Unix platforms by disabling it and going back to the legacy C++ Driver. However, that would get the same tag of the prebuilt official linux toolchain for Fedora that I was using to cross-compile swift-syntax to hang, pegging a CPU core in my Fedora x86_64 virtual server for half an hour each time till I think the host kept killing it.

Only once I put in another workaround for the swift-driver issue and went back to swift-driver did that finally cross-compile, implying the swift-syntax build now depends on some new swift-driver flags that aren't in the legacy C++ Driver anymore, which is all you'll have in this bootstrap compiler too.

Rather than fixing potentially dozens of bootstrap issues like this, I think it would be better to rip out the last remaining bootstrap support and make it easier to cross-compile and test the Swift stdlib and compiler for new platforms, which after all you've been warning people is the plan now for years on the forum.

@DougGregor
Copy link
Member Author

Is this just an experiment to see how far you get or do you really expect to get bootstrapping to work again? It's been a year since the CI switched over to only using a prebuilt Swift 5.8.1 compiler to build the trunk compiler, so I doubt you can get all those dependencies weeded out again.

I'm trying to see whether we can build enough of a toolchain from just the C++ code to serve as a host toolchain to build the full compiler.

Even if you get a basic bootstrap C++-only compiler built again, it may no longer work to build the full compiler with swift-syntax.

To be clear, I don't want to do the bootstrapping mode ever again. It's "use C++ to build a minimal host toolchain, use that as host tools to build the full compiler." If it doesn't work, fine, If it does work, we should consider keeping that around until...

make it easier to cross-compile and test the Swift stdlib and compiler for new platforms, which after all you've been warning people is the plan now for years on the forum.

Yes, that's the goal. We're not at that goal, and are in an unfortunate place where both paths are hard. I'm trying to determine whether we can resurrect the starting-from-C++ path and keep it around a little longer to bridge the gap until cross-compilation becomes easier.

@DougGregor
Copy link
Member Author

Okay, we timed out in testing, so... that's an improvement?

@DougGregor
Copy link
Member Author

preset=buildbot_linux,without_host_swift
@swift-ci Please test with preset Linux Platform

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants